The present invention generally relates to a portable projection device. More specifically, the present invention relates to an apparatus and a method for automated visual distortion adjustments for a portable projection device.
Graphics and video projectors have become popular and cost effective for mass consumer markets in recent years. Once bulky and expensive in the past, projection devices are increasingly becoming miniaturized and less expensive. For example, a standalone projector in the consumer market today is gradually getting lighter, cheaper, and smaller due to projection technology advancements such as miniaturized Digital Light Processing (DLP) technologies. Furthermore, some projection devices are becoming sufficiently miniaturized to be integrated into mobile devices such as cellular phones, laptops, and dedicated portable gaming devices.
Although integration of miniature projection devices into mobile devices are still early in industry adoption cycles, the advantages of projecting graphics on a blank wall or a projection screen from a mobile device may be numerous. For example, a gamer who was previously constrained by a size of a display unit attached to a dedicated portable gaming device or a cellular phone for viewing game-related graphics may find a “blown-up,” large-screen projection of gaming graphics from the dedicated portable gaming device very appealing. The blown-up, large-screen projection capability from a mobile device may even replace a need for a standalone game console device in the future, as the portability of a mobile gaming device and the flexibility of large-screen projection are merged into one lightweight device. Furthermore, the integration of a projection device into a mobile device may empower a user to utilize a large-screen projection in business presentations, conferences, and work-related software applications without being visually limited by a small screen size on a typical mobile device.
In both standalone projector devices and mobile devices with integrated projection capabilities, visual distortions introduced by non-ideal angles between a projecting medium (e.g. a projector lens) and a projected surface (e.g. a blank wall or a projection screen) can cause user inconveniences. For example, a projector lens tilted upward relative to a flat horizontal surface when a projected surface is vertical to the flat horizontal surface introduces a “keystone” visual distortion, with an upper portion of a projected image wider than a lower portion of the projected image. Similarly, if the projector lens is tilted to the right (e.g. clockwise) relative to its non-rotational angle (e.g. in case a surface below the projector lens is tilted, or in case the projector device is held by hand with a slight rotational twist), the projected image also appears “rotated” clockwise or twisted to the right.
There have been some attempts to provide some level of automated adjustments to visual distortions from a projector. U.S. patent application Ser. No. 12/206,554 titled “Method and Apparatus for Depth Sensing Keystoning” by Brandon Dean Slack et al. discloses a method for sensing depth differences of two points on a projected surface, which involves real-time calculations for a multiple-stage graphical transformations of an original image based on a camera image capture or distance-measurement sensor usage. After the multiple-stage graphical transformations involving real-time calculations for transformations, a “corrected” image with adjusted keystone effects can be projected to a screen. Similar to other existing image capture-based adjustment methods, Brandon Dean Slack et al's Ser. No. 12/206,554 requires a heavy amount of computations to derive real-time graphical transformations based on sensor readings. The computational load on a graphics processor or a microprocessor can be especially heavy, if some of the sensors used require an image capture by a camera and image interpretations for graphical transformation calculations (as in FIG. 3A and FIG. 4 of Brandon Dean Slack et al.'s patent application Ser. No. 12/206,554). Heavy computational loads can result in significant processing delay, often at least several seconds from initial detection to automatic adjustments, for each image adjustment.
Another US Patent Application by Brandon Dean Slack, U.S. Ser. No. 12/206,546, discloses a projection system which can be calibrated based on camera image capture and/or an “auxiliary” device, wherein the auxiliary device provides its own perspective or its point of view to the projected image. As stated in Paragraph [0046] of U.S. Ser. No. 12/206,546, the “auxiliary” device is described to be a separate unit from a projection system or a presentation system to provide a camera image capture of a projected screen or other sensory measurements. U.S. Ser. No. 12/206,546 describes the auxiliary device as a sensor device or a client device located somewhere distant from the projection system unit, wherein a main purpose of the auxiliary device is to provide a visual perspective and sensory feedback from the point of view at the location of the auxiliary device to the projection system located somewhere else (i.e. FIG. 8 and Paragraph [0089] in U.S. Ser. No. 12/206,546).
In practical implementations, each step of manipulating images such as image capture, captured image processing, captured image interpretations with other images, and real-time graphic transformation calculations, as disclosed in prior publications such as U.S. Ser. No. 12/206,554 and U.S. Ser. No. 12/206,546, adds a significant amount of time delay before a corrected image can be projected to a projected screen. These conventional image adjustment procedures for projection systems can easily take five to ten seconds, if not longer, before projecting the corrected image. Any multi-second delays are unacceptable if projection image adjustments need to be made continuously in every fraction of a second at a high refresh rate. For example, if an projection image adjustment requires fast refresh rates (e.g. 10˜20 Hz, or 10˜20 times per second for visual distortion adjustment between a projector and a projection screen), the conventional automated projected image adjustment methods such as the ones disclosed in U.S. Ser. No. 12/206,554 and U.S. Ser. No. 12/206,546 are grossly unsuitable due to a heavy processor calculation load and image capture, process, and interpretation workload.
The two prior publications (U.S. Ser. No. 12/206,554, U.S. Ser. No. 12/206,546) by Brandon Dean Slack et al. may be useful for a standalone projector system on a fixed location or a stationary surface, which allows a projection system designer to assume that a calibration or an adjustment of a projector system is typically a one-time event at the beginning of each projector use. However, these methods disclosed by Brandon Dean Slack et al. do not anticipate, suggest, or teach a method or an apparatus to enable a very rapid adjustment, or a very rapid preemptive adjustment for image projection when a projector itself is placed on a non-stationary surface, such as a user's hand which may tilt, vacillate, or move abruptly and frequently.
Because most projection devices are still envisioned and designed to be placed on a stationary flat surface, fast-refresh, real-time, and/or automated adjustments schemes to counteract tilts, rotations, keystones, or other visual distortions caused by an unstable surface (e.g. a projector on a surface of a user's hand) for a projection device are not yet available today. Most mobile devices operate for extended periods on batteries and use microprocessors and graphic chipsets with calculation capabilities typically less than those used in conventional desktop computers. Therefore, for mobile devices which integrate miniaturized projectors, or for highly-portable miniaturized projectors, a novel method and a novel apparatus for rapid visual distortion adjustments which consume low power and exhibit low microprocessor calculation loads may be highly beneficial.